Digital photographers capture image records with digital still cameras, video cameras, camera phones, and other random access digital capture devices. The captured record is initially stored on the capture device and is commonly then moved to personal computer disk memory or online storage systems. Whether the image records are stored on the device or on larger computer systems, the photographer can either manually or automatically organize their image records in a hierarchical fashion into digital content containers (typically called albums or folders). These containers can contain image records and other containers creating a hierarchical storage scheme. Organizing image records by real-life events, such as birthdays, holiday parties, and the like, is one of the most common organization methods used by digital photographers.
When searching and browsing hierarchical image record collections, digital capture devices, personal computer file systems, image organization applications, and online storage systems typically represent a collection of image records with an icon and/or a small-scaled image from the collection usually called a “thumbnail”. The thumbnail image gives the user a view of one image from a potentially large collection of image records to assist them in recognizing the event and content of the collection and is advantageous for this purpose over an icon and collection name. The specific image record selected from the collection to represent the collection is sometimes referred to as a “key” image record. The same approach is utilized to provide representative images, also referred to as “key frames”, of video image records.
Using the wide range of digital capture devices that are available, it is possible for a user to generate and store a sizable collection of image records in digital form. A number of solutions have been proposed for helping the user to sort and select images of most interest from a larger collection. Among these solutions are methods for classifying and grouping images according to event-based probabilistic criteria. For example, commonly assigned U.S. Pat. No. 6,606,411 entitled METHOD FOR AUTOMATICALLY CLASSIFYING IMAGES INTO EVENTS and U.S. Pat. No. 6,351,556 entitled METHOD FOR AUTOMATICALLY COMPARING CONTENT OF IMAGES FOR CLASSIFICATION INTO EVENTS both to Loui et al. disclose various event-based solutions for classifying and grouping image records.
Users tend to capture image records episodically, reflecting different occurrences. Grouping image records to reflect different episodes can be based on spatio-temporal differences, such as time or distance, or, in a more sophisticated manner, based upon event and subevent. These approaches tend to be convenient for many people, but have the shortcoming that the subject matter of the image records in groups and subgroups is not necessarily apparent from the grouping procedure. A representative image record can make the subject matter of a group or subgroup apparent, but is relatively difficult to determine. This is unlike groupings, in which all members necessarily have the same subject matter, such that a member of a group is representative of the group. For example, any member of the group “pictures of the new baby” would be capable of representing the group as a picture of the new baby.
Many systems use the first image record in a set of image records as a representative image record. The first image record is typically chronologically the earliest. The selection of the first image record often does not adequately reflect the context or content of the other image records in the set. For example, many digital photographers capture content before an actual event begins simply to verify camera operation. The content captured for this purpose is arbitrary and may or may not reflect the content captured during the actual event. Actual event content captured at the beginning of a lengthy event also frequently does not accurately reflect the context and setting of the entire event.
U.S. Pat. No. 6,847,733 to Savakis et al. discloses a method in which images are grouped and representative images of the groups are determined based upon an understanding of the content (semantic saliency features) in the images. U.S. Pat. No. 6,721,454 to Qian et al. discloses a method in which video sequences are analyzed to determine semantic and other saliency features that are then summarized in textual descriptions.
While a number of methods for automated image classification and grouping have been developed, however, the task of selecting image records from the larger collection is inherently more difficult. Using a simple event-based model, a subset of image records can be obtained from a larger collection by taking a representative image record from each event-based grouping, for example. However, this type of selection would not be appropriate for a number of situations in which a user may want to obtain, from a large collection of image records, a small subset of image records of a subject of interest according to a temporal sequence. Thus, for example, a user may want to obtain, from a large image-record collection, a subset of image records that show significant stages or events in the life of a friend or family member. Conventional methods for automated selection of a subset of image records from the full collection could identify events with some granularity; however, there are some periods in the person's life that are generally of more interest than others. With a family member, for example, image records from childhood may be of considerable interest, whereas image records taken during adulthood may be of less interest.
Thus, it can be appreciated that a need in the art exists for improved or additional image-record-selection techniques.